Convex Relaxations for Learning Bounded-Treewidth

Derivation of Cost Function. The projection of the joint probability distribution of the random variables X = (X1, X2, . . . , Xn), associated with the vertices in V , on a decomposable graph G is given by: pG(x) = ∏ C∈C(G) pC(xC) ∏ (C,D)∈T (G) pC∩D(xC∩D) , (1) where x is an instance in the domain of X, which we denote by X . pC(xC) denotes the marginal distribution of random variables belonging to C ∈ C(G) and pC∩D(xC∩D) denotes the marginal distribution of random variables belonging to the separator set C ∩D, such that (C,D) ∈ T (G). Let p̂(x) denote the empirical distribution and p̂G(x) denotes the projected distribution on a decomposable graph G. Estimating the maximum likelihood decomposable graph which best approximates p̂ is equivalent to finding the graph, G, which minimizes the KL-divergence between the target distribution and the projected distribution, p̂G, defined by D(p̂||p̂G). D(p̂||p̂G) = ∑ x∈X p̂(x) log p̂(x) p̂G(x) ∝ ∑ x∈X −p̂(x) log p̂G(x) as p̂(x) is independent of G,